Live migration of virtual machines (VMs) is a critical activity in the operation of modern data centers. Live migration involves the transfer of multiple Gigabytes of memory within a short duration (assuming that network attached storage is used, which does not require migration) and can consequently consume significant amounts of network and CPU resources.
An administrator may need to simultaneously migrate multiple VMs to perform resource re-allocation to handle peak workloads, imminent failures, cluster maintenance, or powering down an entire rack to save energy, Simultaneous live migration of multiple VMs is referred to as gang migration[8]. Gang migration is a network intensive activity that can cause an adverse cluster-wide impact by overloading the core links and switches of the datacenter network. Gang migration can also affect the performance at the network edges where the migration traffic competes with the bandwidth requirements of applications within the VMs. Hence it is important to minimize the adverse performance impact of gang migration by reducing the total amount of data transmitted due to VM migration. Reducing the VM migration traffic can also lead to a reduction in the total time required to migrate multiple VMs.
Process migration has also been extensively researched. Numerous cluster job schedulers exist, as well as virtual machine management systems, such as VMWare's DRS, XenEnterprise, Usher, Virtual Machine Management Pack, and CoD that let administrators control jobs/VM placement based on cluster load or specific policies such as affinity or anti-affinity rules.
[27] optimizes the live migration of a single VM over wide-area network through a variant of stop-and-copy approach which reduces the number of memory copying iterations. [30] and [27] further use page-level deduplication along with the transfer of differences between dirtied, and original pages, eliminating the need to retransmit the entire dirtied page. [16] uses an adaptive page compression technique to optimize the live migration of a single VM. Post-copy [13] transfers every page to the destination only once, as opposed to the iterative pre-copy[20], [5], which transfers dirtied pages multiple times. [14] employs low-overhead RDMA over Infiniband to speed up the transfer of a single VM. [21] excludes the memory pages of processes communicating over the network from being transferred during the initial rounds of migration, thus limiting the total migration time. [29] shows that certain benchmarks used in high performance computing are likely to have large amounts of content sharing. The work focuses mainly on the opportunity and feasibility of exploiting content sharing, but does not provide an implementation of an actual migration mechanism using this observation, nor does it evaluate the migration time or network traffic reduction. Shrinker[22] migrates virtual clusters over high-delay links of WAN. It uses an online hashing mechanism in which hash computation for identifying duplicate pages (a CPU-intensive operation) is performed during the migration.
The following US patents and published patent applications are expressly incorporated herein in their entirety: 20130339407; 20130339390; 20130339310; 20130339300; 20130339299; 20130339298; 20130332685; 20130332660; 20130326260; 20130326159; 20130318051; 20130315260; 20130297855; 20130297854; 20130290267; 20130282662; 20130263289; 20130262801; 20130262638; 20130262615; 20130262410; 20130262396; 20130262394; 20130262392; 20130262390; 20130262386; 20130262385; 20130254402; 20130253977; 20130246366; 20130246360; 20130238575; 20130238572; 20130238563; 20130238562; 20130232215; 20130227352; 20130212437; 20130212200; 20130198459; 20130159648; 20130159645; 20130151484; 20130138705; 20130132967; 20130132531; 20130125120; 20130121209; 20130117240: 20130111262; 20130110793; 20130110779: 20130110778; 20130097380; 20130097377; 20130086353; 20130086269; 20130086006; 20130080728; 20130080408; 20130061014; 20130055249; 20130055248; 20130054932; 20130054927; 20130054910; 20130054906; 20130054890; 20130054889; 20130054888; 20130054545; 20130046949; 20130042052; 20130041872; 20130031563; 20130031331; 20130024645; 20130024424; 20120331021; 20120290950; 20120284236; 20120254119; 20120240110; 20120239871; 20120213069; 20120102455; 20120089764; 20120084595; 20120084527; 20120084507; 20120084506; 20120084505; 20120084504; 20120084270; 20120084262; 20120084261; 20120079318; 20120079190; 20120079189; 20120017114; 20120017027; 20120011176; 20110238775: 20110179415; 20110167221; 20110161723: 20110161299; 20110161297; 20110161295; 20110161291; 20110087874; 20100333116; 20100332818; 20100332658; 20100332657; 20100332479; 20100332456; 20100332454; 20100332401; 20100274772; 20100241807; 20100241726; 20100241673; 20100241654; 20100106691; 20100070725; 20100070528; 20100011368; 20090240737; 20060069717; 20060010195; 20050262194; 20050262193; 20050262192; 20050262191; 20050262190; 20050262189; 20050262188; 20050240592; 20050240354; 20050235274; 20050234969; 20050232046; 20050228808; 20050223109; 20050222931; U.S. Pat. Nos. 8,612,439; 8.601,473; 8,600,947; 8,595,460; 8,595,346; 8,595,191; 8,589,640; 8,577,918; 8,566,640; 8,554,918; 8,549,518; 8,549,350; 8,549,245; 8,533,231; 8,527,544; 8,516,158; 8,504,870; 8,504,791; 8,504,670; 8,489,744; 8,484,505; 8,484,356; 8,484,249; 8,463,991; 8,453,031; 8,452,932; 8,452,731; 8,442,955; 8,433,682; 8,429,651; 8,429,649; 8,429,360; 8,429,307; 8,413,146; 8,407,428; 8,407,190; 8,402,309; 8,402,306; 8,375,003; 8,335,902; 8,332,689; 8,311,985; 8,307,359; 8,307,177; 8,285,681; 8,239,584; 8,209,506; 8,166,265; 8,135,930; 8,060,553; 8,060,476; 8,046,550; 8,041,760; 7,814,470; and 7,814,142.